Eeciency Considerations in Very Large Information Retrieval Servers

It is estimated that the World Wide Web now contains more than twenty million diierent content areas, presented on more than 320 million web pages, and one million web servers| and it is doubling every nine months 16, 17]. To combat this, Moore's law suggests that computational resource will continue to double every eighteen months. This suggests that| although both curves are exponential|we may be losing ground in the quest to search and nd useful information. We brieey review some of the algorithms known to be useful for very large scale information retrieval and suggest opportunities for improvement in this area. Without research in this area, the web and other electronic information sources will become large haystacks in which no one will be able to nd any needles.